Search for: All records

There is a growing community of researchers at the intersection- tion of data mining, AI, and computing education research. The objective of the CSEDM workshop is to facilitate a dis- Discussion among this research community, with a focus on how data mining can be uniquely applied in computing ed- ucation research. For example, what new techniques are needed to analyze program code and CS log data? How do results from CS education inform our analysis of this data? The workshop is meant to be an interdisciplinary event at the intersection of EDM and Computing Education Research. Researchers, faculty, and students are encouraged to share their AI- and data-driven approaches, methodological- gies, and experiences where data transforms how students learn Computer Science (CS) skills. This full-day workshop will feature paper presentations and discussions to promote collaboration. 

Teachers often use open-ended questions to promote students' deeper understanding of the content. These questions are particularly useful in K–12 mathematics education, as they provide richer insights into students' problem-solving processes compared to closed-ended questions. However, they are also challenging to implement in educational technologies as significant time and effort are required to qualitatively evaluate the quality of students' responses and provide timely feedback. In recent years, there has been growing interest in developing algorithms to automatically grade students' open responses and generate feedback. Yet, few studies have focused on augmenting teachers' perceptions and judgments when assessing students' responses and crafting appropriate feedback. Even fewer have aimed to build empirically grounded frameworks and offer a shared language across different stakeholders. In this paper, we propose a taxonomy of feedback using data mining methods to analyze teacher-authored feedback from an online mathematics learning platform. By incorporating qualitative codes from both teachers and researchers, we take a methodological approach that accounts for the varying interpretations across coders. Through a synergy of diverse perspectives and data mining methods, our data-driven taxonomy reflects the complexity of feedback content as it appears in authentic settings. We discuss how this taxonomy can support more generalizable methods for providing pedagogically meaningful feedback at scale. 

Novice programmers often face challenges in designing computational artifacts and fixing code errors, which can lead to task abandonment and over-reliance on external support. While research has explored effective meta-cognitive strategies to scaffold novice programmers' learning, it is essential to first understand and assess students' conceptual, procedural, and strategic/conditional programming knowledge at scale. To address this issue, we propose a three-model framework that leverages Large Language Models (LLMs) to simulate, classify, and correct student responses to programming questions based on the SOLO Taxonomy. The SOLO Taxonomy provides a structured approach for categorizing student understanding into four levels: Pre-structural, Uni-structural, Multi-structural, and Relational. Our results showed that GPT-4o achieved high accuracy in generating and classifying responses for the Relational category, with moderate accuracy in the Uni-structural and Pre-structural categories, but struggled with the Multi-structural category. The model successfully corrected responses to the Relational level. Although further refinement is needed, these findings suggest that LLMs hold significant potential for supporting computer science education by assessing programming knowledge and guiding students toward deeper cognitive engagement. 

Abstract Music and computer science (CS) have profound historical and structural connections, with programming music offering a promising avenue for engaging children in CS through creative expression. To foster this engagement, our team developed M-Flow, a flow-based music programming platform designed to introduce students to CS via music. Despite extensive existing research in music and CS education, experience reports and empirical studies on K-12 teachers' implementation and its impact on young kids' learning are limited. Therefore, we recruit elementary school teachers and students with no or limited prior programming experience, introducing them to M-Flow and its curriculum through a professional development workshop, a semester's job embedded support, and classroom implementation. We describe the experiences of teachers as they attempt to integrate music and CS, the challenges they face, and the influence on students' attitudes toward learning computing concepts. Specifically, we reflect on our intervention by conducting a sequential mixed-method evaluation. During the qualitative phase, we collected multiple sources of data from three teachers through focus groups and debriefings after a semester of classroom implementation. Thematic analysis of workshop activities, interviews, and debrief videos revealed three themes with seven sub-themes on teachers' integration of flow-based music programming and two themes with five sub-themes on challenges faced by the teachers. In the quantitative phase, we gathered data on attitudes and self-efficacy from 75 students taught by these teachers. Results indicate that the flow-based music programming environment provided an engaging programming experience for students and significantly increased their self-efficacy towards learning programming. 

Pedagogical agents (PAs) are increasingly being integrated into educational technologies. Although previous reviews have examined the impact of PAs on learning and learning-related outcomes, it still remains unclear what specific design features, social cues, and other contextual elements of PA implementation can optimize the learning process. These questions are even more prevalent with regards to the K-12 population, as most reviews to date have largely focused on post-secondary learners. To address this gap in the literature, we systematically review empirical studies around the design of PAs for K-12 learners. After reviewing 1374 studies for potential inclusion, we analyzed 44 studies that met our inclusion criteria using Heidig and Clarebout’s (2011) frameworks. Our findings showed that learners had preferences for specific types of PAs. While these preferences were not always associated with increased learning outcomes, there is a lack of research specifically investigating the intersection of perceptions and learning. Our results also showed that pedagogical strategies that are effective for human teachers were effective when used by PAs. We highlight what specific design features instructional designers can use to design PAs for K-12 learners and discuss promising research directions based on the extant work in the field. 

Systematic reviews are a time-consuming yet effective approach to understanding research trends. While researchers have investigated how to speed up the process of screening studies for potential inclusion, few have focused on to what extent we can use algorithms to extract data instead of human coders. In this study, we explore to what extent analyses and algorithms can produce results similar to human data extraction during a scoping review—a type of systematic review aimed at understanding the nature of the field rather than the efficacy of an intervention—in the context of a never before analyzed sample of studies that were intended for a scoping review. Specifically, we tested five approaches: bibliometric analysis with VOSviewer, latent Dirichlet allocation (LDA) with bag of words, k-means clustering with TF-IDF, Sentence-BERT, or SPECTER, hierarchical clustering with Sentence-BERT, and BERTopic. Our results showed that topic modeling approaches (LDA/BERTopic) and k-means clustering identified specific, but often narrow research areas, leaving a substantial portion of the sample unclassified or in unclear topics. Meanwhile, bibliometric analysis and hierarchical clustering with SBERT were more informative for our purposes, identifying key author networks and categorizing studies into distinct themes as well as reflecting the relationships between themes, respectively. Overall, we highlight the capabilities and limitations of each method and discuss how these techniques can complement traditional human data extraction methods. We conclude that the analyses tested here likely cannot fully replace human data extraction in scoping reviews but serve as valuable supplements. 

Question-asking is a crucial learning and teaching approach. It reveals different levels of students' understanding, application, and potential misconceptions. Previous studies have categorized question types into higher and lower orders, finding positive and significant associations between higher-order questions and students' critical thinking ability and their learning outcomes in different learning contexts. However, the diversity of higher-order questions, especially in collaborative learning environments. has left open the question of how they may be different from other types of dialogue that emerge from students' conversations, To address these questions, our study utilized natural language processing techniques to build a model and investigate the characteristics of students' higher-order questions. We interpreted these questions using Bloom's taxonomy, and our results reveal three types of higher-order questions during collaborative problem-solving. Students often use Why, How and What If' questions to I) understand the reason and thought process behind their partners' actions: 2) explore and analyze the project by pinpointing the problem: and 3) propose and evaluate ideas or alternative solutions. In addition. we found dialogue labeled 'Social'. 'Question - other', 'Directed at Agent', and 'Confusion/Help Seeking' shows similar underlying patterns to higher-order questions, Our findings provide insight into the different scenarios driving students' higher-order questions and inform the design of adaptive systems to deliver personalized feedback based on students' questions. 

Over the past three decades the field of pedagogical agents (PAs) has seen significant growth, but no review has specifically focused on the design and use of PAs for K-12 students, despite the fact that an early meta-analysis showed that they receive the most benefits from learning from or with PAs. Our systematic search revealed 112 studies that met the inclusion criteria and were analyzed. Our findings revealed a plethora of studies investigating the use of PAs with K-12 populations and a considerable number of longitudinal studies, both of which the field has long stated did not exist in significant numbers. Our findings contrast long-held findings in the field, further support others, and highlight areas where further experimentation and research synthesis are needed. 

K-12 Computer Science (CS) education has seen remarkable growth recently, driven by the increasing focus on CS and Computational Thinking (CT) integration. Despite the abundance of Professional development (PD) programs designed to prepare future CS teachers with the required knowledge and skills, there is a lack of research on how teachers' perceptions and attitudes of CS and CT evolve before and after participating in these programs. To address this gap, our exploratory study aims to study the dynamics of pre-and in-service teachers' experiences, attitudes, and perceptions towards CS and CT through their participation in a K-12 CS education micro-credential program. In this study, we employed topic modeling to identify topics that emerged from teachers' written pre- and post-CS autobiographies, conducted statistical analysis to explore how these topics evolve over time and applied regression analysis to investigate the factors influencing these dynamics. We observed a shift in teachers' initial feelings of fear, intimidation, and stress towards confidence, fun, and feeling competent in basic CS, reflecting a positive transformation. Regression analysis revealed that features, such as experienced teacher status and CT conceptual understanding, correlate with participants' evolving views. These observed relationships highlight the micro-credential's role in not only enhancing technical competency but also fostering an adaptive, integrative pedagogical mindset, providing new insights for course design.